Human Path Finding in a Semantic Word Game

Human decision-making processes are inevitably influenced by mental images, which are triggered by cues such as objects, concepts, and words. The cognitive associations between words can be captured and represented as a semantic network, in which words represent nodes and association between words are edges. How efficiently can humans traverse this word association network to a given target word if, at each moment, they can only make choices about the next word they will visit? Using participant traversal data from the appropriately designed semantic game MindPaths, we show that human players do not rely on guessing, but navigate this association network quite efficiently, finding a specified target word often in the minimal number of steps. We construct models to capture human paths within the MindPaths game. We find that similarity in overall game length is easy to achieve with a trained random walker. We then consider a model of individual choices using Bayesian estimation of transition probabilities conditional on local network connectivity. We find that it is difficult to capture the individual decisions or the decision making process of human players. More complex models driven by semantic similarities and other types of relations between words are necessary to fully understand how individuals choose a particular path within this semantic word game. Introduction How well do we know our own language? The ubiquity of the internet and the advent of blogs and social media have led to an exponentially growing quantity of information created by an emergent culture of blogging, liking and commenting. The ability to read and critically question content have become crucial skills for competent internet users. It has long been recognized by artists as well as propagandists likewise that words invoke feelings and cognitive associations which can easily be used and abused. The cognitive connections between words span a semantic association network, a so-called semantic space. The network is malleable by nature and new associations can be formed as regularly applied by, e.g., marketing and election slogans. In this paper, we analyze the ability of individuals to intentionally represent and navigate a semantic network representation when given a random start and a specific target. By constraining the full complexity of an individual’s semantic space to a predefined network, we explore and model whether individual route choices are based on strategic decisions. Previous research on semantic space suggests that people have an intuitive idea about how “far“ apart two words are in terms of semantic similarity and that a network is a useful and predictive representation of semantic information1, 2. Furthermore, experimental evidence has found that the ability to navigate a semantic space is related to cognitive ability more broadly3, 4. Additionally, vector representation models such as Word2Vec5 have shown that distance between vectors is strongly related and even predictive of similarity judgments of individuals. To address the question of semantic space navigation, a type of information foraging, a semantic word game has been conceived in which players have to reach a target word starting from a random word, moving along an underlying semantic network composed of words6. This game has further been adapted to an online platform, allowing for more than 800 individuals in many different countries to participate in the game1. The underlying network is composed of words as nodes and directed edges, which represent cognitive associations according to the University of South Florida’s free association norms database (see Methods). Links can be one-directional, such as bone← dog or bidirectional, e.g. dog↔ cat, if one word is a response to the cue word and vice versa. Shortest paths from target to end word are not unique and may follow different intuitions, as illustrated by team→ f ootball→ kick→ scream→ horror and team→ coach→ stage→ f right→ horror. While it is unlikely that the underlying semantic association network used in this game captures the full richness and uniqueness of an individual’s semantic network, performance within this game can nonetheless provide researchers with an understanding about the type of information people access when making a choice of what word to move to next. We 1mindpaths.socientize.eu explore whether players are using some underlying strategy for their decision-making process and whether their choices can be explained in terms of pure network topology or similarity in semantic space. Can such network measures as eigencentrality and in-degree capture how individuals are making specific choices? Or does more informed semantic similarity such as those based on distances in vector space representations better account for performance of individuals in this navigation task? We construct a Bayesian model based on the former assumption and assess our model on the ability to capture and predict individual human choices and collective behavior in this task. The rest of the paper is organized as follows. We first analyze the overall performance of the human players in the semantic word game and underline how the number of choices to successful navigation increases as the distance between start and end words increases. Subsequently, we introduce a benchmark model for the decision-making modeling, namely an unbiased random walker, and show that the players’ strategies are clearly not based on mere guessing. We then investigate a trained random walker model which manages to replicate the general trends of human performance quite well, having a higher success rate on trials that require fewer intermediate word choices. However, we observe a discrepancy regarding individual word choices in particular games. The trained random walker will choose, in the limit, each word that decreases the distance to the target word with equal probability, whereas the human players seem to have a preference for specific paths which take them to the target word. Finally, we introduce a Monte-Carlo Markov-Chain (MCMC) modeling framework which expresses the transition probabilities from one word to another in terms of structural features of the network. For example, our Bayesian model may learn to favor high degree words when the target is far away and then narrow down on the specific goal word by means of eigencentrality. We conclude from the performance of this class of models that different aspects of the network seem to be of more or less importance for the decision-making and for the success of individuals. Results Our work is based on the data collected from the word game MindPaths2. This game is based on a prior research experiment discussed in Beckage, Butts and Steyvers 20127. In this game, individuals are asked to navigate from a start word to an end word through a series of discrete choices on a predefined semantic network consisting of 2392 unique words and 23,571 directed link. Each link represents a free association between two words (see Methods). The distance from start to end word could take a value between one, in which the player can directly select the target word, to a maximum of six. Only trials where individuals reach the end word in less than 25 steps are included in our analysis, since reasons for unsuccessful plays are many faceted and can include uncontrollable factors such interruption of internet connection. See7 for an analysis exploring the overall rate of success in this type of semantic game. For interested readers, other work has focused explicitly on why individuals give up in a similar network navigation task8. The original data contains information on the decision times in milliseconds, which we neglect here but which offers possibilities for future research. Game design and Vocabulary For ease of reference to important states of the game, we briefly introduce a few terms. The target word refers to the word that an individual is trying to find. If this word is selected the game ends. The geodesic distance between two nodes is the minimum number of steps required to reach one node starting from the other, i.e. the length of a shortest path between these nodes on the network. The total number of moves which a player uses to solve a game is referred to as the game length. We define the current word as either the start word or the most recent word selected by an individual. The option set are the neighbors that the current word points to. The option set is constrained to be between 3 and 12 words. If a particular word had more than 12 out-neighbors, the strongest 12 were presented. Words with less than 3 associates were not included in the game network. We define the choice to be the specific word from the option set that was selected by an individual or a model. Fig. 1 shows a screen shot of the game. Human Performance As shown in Fig. 2, the players perform surprisingly well in the semantic games, solving games in the minimum number of steps in approximately 33% of the successful trials. This indicates that they were actively searching to reach the target word quickly and that words lying on the shortest paths met their ideas of being effective moves. In some cases, however, players require significantly more steps. The increase in game length for particular individuals or games could be due to a simple guessing strategy or to the fact that the network representation is not in line with the players’ expectations. In fact, navigation to some of the targets along the geodesic do sometime require topic shifts. One such example is the example from above where individuals search for a path of team→ . . .→ horror. In this case, the start and end words are thematically very different. Another common feature of these games is that the shortest path may require the use of polysemy, or the use of two different meanings of the same orthographic word, to alter the concept space as in the play pillow→ sheet→ paper→ pen. A general 2See mindpaths.socientize.eu for more detail, to play the game and to contribute to ongoing research.